The present invention relates generally to visual imaging systems, and more particularly to visual imaging systems and techniques which provide useful electronic manipulation of wide angle hemispheric scenes.
The collection, storage, and display of large areas of visual information can be an expensive and difficult process to achieve accurately. With the recent increased emphasis on multimedia applications, various methods and apparatus have been developed to manage visual data, A unique class of multimedia data sets is that of hemispheric visual data. Known multimedia methods and apparatus attempt to combine various multimedia imaging data, such as still and motion (or video) images, with audio content using storage media such as photographic film, computer diskettes, compact discs (CDs), and interactive CDs. These are used in traditional multimedia applications in various fields, such as entertainment and education. Non-multimedia applications also exist that would employ hemispheric visual data, such as in security, surveillance, unmanned exploration, and fire and police situations. However, as will be described below, the known methods and apparatus have certain limitations in capturing and manipulating valuable information of hemispheric scenes in a rapid (i.e., real-time) and cost-effective manner.
One well-known multimedia technique is used at theme parks, wherein visual information from a scene is displayed on a screen or collection of screens that covers almost 360 degrees field of view. Such a technique unfortunately results in the consumption of vast quantities of film collected from multiple cameras, requires specially-designed carriages to carry and support the cameras during filming of the scene, and necessitates synchronization of shots during capture and display. The technique is also limited in that the visual image cannot be obtained with a single camera nor manipulated for display, e.g., pan, tilt, zoom, etc., after initial acquisition. Hence, this technique, while providing entertainment, is unable to fulfill critical technical requirements of many functional applications.
Other known techniques for capturing and storing visual information about a large field of view (FOV) are described in U.S. Pat. Nos. 4,125,862; 4,442,453; and 5,185,667. In U.S. Pat. No. 4,125,862, a system is disclosed that converts signal information from a scene into digital form, stores the data of the digitized scene serially in two-dimensional format, and reads out the data by repetitive scan in a direction orthogonally related to the direction in which the data was stored. U.S. Pat. No. 4,442,453 discloses a system in which a landscape is photographed and stored on film. The film is then developed, with display accomplished by scanning with electro-optical sensors at "near real-time" rates. These techniques, however, do not provide instant visual image display, do not cover the field of view required for desired applications (hemispheric or 180 degrees field of view), do not generate visual image data in the format provided by the techniques of this invention, and are also not easily manipulated for further display, e.g., pan, tilt, etc.
The technique disclosed in the U.S. Pat. No. 5,185,667 overcomes some of the above-identified drawbacks in that it is able to capture a near-hemispheric field of view, correct the image using high speed circuitry to form a normal image, and electronically manipulate and display the image at real-time rates.
For many hemispheric visual applications, however, a system of the type described by the '667 patent has limitations in obtaining sufficient information of critical and useful details. This is particularly true when the camera is oriented with the central axis of the lens perpendicular to the plane bounding the hemisphere of acquisition (i.e. lens pointing straight up). In such applications, the majority of critical detail in a scene is contained in areas of the field along the horizon and little or no useful details are contained in central areas of the field located closer to the axis of the lens (the horizon being defined as the plane parallel to the image or camera plane and perpendicular to the optical axis of the imaging system). For example, in surveillance, the imaging system is aimed upward and the majority of the critical detail in the scene includes people, buildings, trees, etc.--most of which are located within only a few degrees along the horizon (i.e., this is the peripheral content). Also, in this example, although the sky makes up the larger central area of the view, it contains little or no useful information requiring higher relative resolution.
To obtain sufficient detail on the critical objects in the scene, the technique should be able to differentiate between the relevant visual information along the horizon and the remaining visual information in the scene in order to provide greater resolution in areas of higher importance. The system described by the '667 patent does not differentiate between this relevant visual information contained along the horizon and the remaining visual information in this scene. Thus, it fails to yield a sufficient quality representation of the critical detail of the scene for projected applications.
Instead, techniques described above concentrate on obtaining, storing, and displaying the entire visual information in the scene, even when portions of this information are not necessary or useful. To obtain the near-hemispheric visual information, such techniques require specific lens types to map image information in the field of view to an image plane (where either a photographic film or electronic detector or imager is placed). Known examples of U.S. Pat. No. 5,185,667 and U.S. Pat. No. 4,442,453, respectively use a fish-eye lens and a general wide-angle lens. As these lenses map information of a large field without differentiation between the central and peripheral areas, information from the periphery will be less fully represented in the image plane than from the central area of acquisition.
U.S. Pat. No. 4,170,400 describes a wide-angle optical system employing a fiber optic bundle that has differing geometric shapes at the imaging ends. Although this is useful in itself for collecting and repositioning image data, bending of light is a natural characteristic of optical fibers and not exclusive to that patent. Further, U.S. Pat. No. 4,170,400 employs a portion of a spherical mirror to gather optical information, rendering a very reduced subset of the periphery in the final imaging result. This configuration is significantly different from the multi-element lens combination described in the present invention.
Imperfections in the image representation of any field inherently result from the nature of creating an image with any spherical glass (or plastic) medium such as a lens. The magnitude of these imperfections increases proportionally to the distance a point in the field is from the axis perpendicular to the optical imaging system. As the angle between the optical axis and a point in the field increases, aberrations of the corresponding image increase proportional to this angle cubed. Hence, aberrations are more highly exaggerated in the peripheral areas with respect to more central areas of a hemispheric image.
Although the lens types above achieve a view of a large field, the valuable content from the peripheral areas lacks in potential image quality (resolution) mapping because the imaging device and system does not differentiate between these areas and the central areas of less valuable detail. Often, the difference between the imaging capabilities between the two areas is compensated for by using only the central portion of a lens to capture the scene ("stopping the lens down"). This works in effect to reduce the image quality of both areas such that the difference in error is a lesser percentage of the smallest area even the central area can resolve. Simultaneously, this compensation technique further degrades the performance of the lens by limiting the amount of light which is allowed to enter the lens, and thus reducing the overall intensity of the image.
More typically, the peripheral content imaged by a conventional lens is so degraded in comparison with the central area that the lens allows for only a minimal area of the periphery to be recorded by the film or electronic imager. As a result of these "off-axis" aberrations inherent to large fields, the relevant information of the horizon in the scene can be underutilized or worse yet--lost.
Another limitation in U.S. Pat. No. 5,185,667 is its organization for recording only views already corrected for perspective. The nature of that methodology is that the specific view of interest must be selected and transformed prior to the recording process. The result is that no additional selection of views can be accomplished after the storage process, reducing system flexibility from the user's perspective.
Hence, there is a demand in the industry for single camera imaging systems that efficiently capture, store, and display valuable visual information within a hemispheric field of view containing particularly peripheral content, and that allow electronic manipulation and selective display of the image post-acquisition while minimizing distortion effects.